Imaging apparatus and image synthesizing method

ABSTRACT

An object of the exemplary embodiment is to make it possible to easily take a group photo of a plurality of persons including a photographer. 
     In a digital camera  1 , an image a of other photographed persons is taken by a photographer immediately after an instruction of a composite shooting mode is given. Next, the digital camera  1  is handed over to one of the other photographed persons, by whom an image b of the photographer is taken. When predetermined synthesizing conditions are satisfied, a facial image contained in one of the two taken images a and b shot in series is synthesized with the other taken image so that a new composite image c is created. Thus, a group photo in which all the persons have been photographed can be synthesized easily.

CROSS REFERENCE TO RELATED APPLICATION(S)

The present disclosure relates to the subject matters contained inJapanese Patent Application No. 2010-255460 filed on Nov. 16, 2010,which are incorporated herein by reference in its entirety.

FIELD

Exemplary embodiments of the present invention relate to an imagingapparatus and an image synthesizing method suitable for taking a groupphoto.

BACKGROUND

When one of a plurality of persons serves as a photographer to take agroup photo of the persons, the photographer cannot be in the groupphoto. Therefore, it has been necessary to ask a third person to take aphoto or to fix a camera to a tripod in order to take a photo with aself-timer.

However, when there is no third person or when there is no tripod, thecamera has to be placed on top of something to take a photo with theself-timer. Due to some location where the camera is placed, it may bedifficult to set an angle of view. Further, to perform shooting with theself-timer, a photographer sets an angle of view to have all the personsexcept the photographer come within a visual field of the camera, pushesa shutter button and then goes into the visual field as a photographedperson quickly. On this occasion, due to some position the photographerenters, the photographer may be out of the set angle of view or thepositions of the photographed persons as a whole may change so that thephotographed persons cannot come within the set angle of view.

Therefore, there has been proposed a technique in which an image ofphotographed persons except a photographer is first taken, an image ofthe photographer is next taken, and the two taken images are suitablysynthesized to create a group photo having all the persons photographed(see JP-A-2005-94741, for instance).

The related-art technique has been described on the assumption that acellular phone which has imaging portions in a front surface and a backsurface of its body is used. A facial region of the photographer isdetected from data of a photographer-side image taken by a first cameraportion. Based on data of a photographed persons-side image taken by asecond camera portion, a region which does not include any photographedpersons is discriminated from regions of the photographed persons. Basedon a result of the discrimination, the detected facial region of thephotographer is synthesized in the region which does not include anyphotographed persons.

SUMMARY

In the aforementioned related-art technique, the photographer and thephotographed persons are shot concurrently by the first camera portionand the second camera portion so that labor for photographing can besaved. On the assumption that a typical digital camera etc. is used,however, there may a problem that provision of two imaging portions,particularly provision of two imaging portions only for taking a groupphoto leads to inefficiency as well as increase in cost.

It is therefore an object of exemplary embodiments of the invention toprovide an imaging apparatus and an image synthesizing method in which agroup photo of a plurality of persons including a photographer can betaken easily.

According to a first aspect of the invention, there is provided animaging apparatus including: an imaging portion which takes images ofsubjects; an instruction portion which gives an instruction tosynthesize two taken images; a first specification portion whichspecifies, as a first taken image, a taken image first shot by theimaging portion after the instruction given by the instruction portionor a taken image last shot by the imaging portion before the instructiongiven by the instruction portion; a second specification portion whichspecifies a second taken image having a predetermined relation to thefirst taken image; a facial image extraction portion which extracts afacial image from one taken image of the two taken images including thefirst taken image and the second taken image; and a synthesizing portionwhich synthesizes the facial image extracted by the facial imageextraction portion with a region of the other taken image of the twotaken images where there is no facial image.

According to a second aspect of the invention, there is provided animage synthesizing method in an imaging apparatus comprising an imagingportion for taking images of subjects, the method including: giving aninstruction to synthesize two taken images; specifying, as a first takenimage, a taken image first shot by the imaging portion after theinstruction is given, or a taken image last shot by the imaging portionbefore the instruction is given; specifying a second taken image havinga predetermined relation to the first taken image; extracting a facialimage from one taken image of the two taken images including the firsttaken image and the second taken image; and synthesizing the extractedfacial image with a region of the other taken image of the two takenimages where there is no facial image.

According to the embodiments, it is possible to gain an advantage that agroup photo of a plurality of persons including a photographer can betaken easily.

BRIEF DESCRIPTION OF THE DRAWINGS

A general configuration that implements the various features of theinvention will be described with reference to the drawings. The drawingsand the associated descriptions are provided to illustrate embodimentsof the invention and should not limit the scope of the invention.

FIG. 1 is a block diagram showing configuration of a digital cameraaccording to a first embodiment of the invention.

FIGS. 2A to 2C are conceptual views showing a fundamental flow to take agroup photo with the digital camera according to the first embodiment.

FIG. 3 is a flow chart for explaining operation of the digital cameraaccording to the first embodiment.

FIG. 4 is a flow chart showing exemplary synthesizing conditions in theoperation of the digital camera according to the first embodiment.

FIG. 5 is a flow chart showing exemplary synthesizing conditions in theoperation of the digital camera according to the first embodiment.

FIG. 6 is a flow chart for explaining the processing of Step S26according to the first embodiment, in which each facial portion isdetected in one taken image and a position where the facial portion willbe embedded in the other taken image is determined.

FIG. 7 is a conceptual view showing an example of operation for decidingan insertion position in a taken image a in the first embodiment.

FIG. 8 is a conceptual view showing the example of operation fordeciding the insertion position in the taken image a in the firstembodiment.

FIG. 9 is a flow chart for explaining operation of a digital cameraaccording to a second embodiment of the invention.

FIG. 10 is a flow chart for explaining processing according to a thirdembodiment of the invention, in which each facial portion is detected inone taken image and a position where the facial portion will be embeddedin the other taken image is determined.

FIG. 11 is a block diagram showing configuration of a digital cameraaccording to a fourth embodiment of the invention.

FIGS. 12A to 12E are conceptual views showing a fundamental flow to takea group photo with digital cameras according to the fourth embodiment.

FIG. 13 is a flow chart for explaining operations of the digital camerasaccording to the fourth embodiment.

FIG. 14 is a sequence diagram for explaining operations of digitalcameras 1-1 and 1-2 according to the fourth embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the invention will be described below with reference tothe drawings.

A. First Embodiment

FIG. 1 is a block diagram showing configuration of a digital cameraaccording to a first embodiment of the invention. In FIG. 1, a digitalcamera 1 has an imaging lens 2, a lens drive portion 3, a shutter 4which also serves as an aperture, a CCD 5, a Timing Generator (TG) 6, aunit circuit 7, an image processing portion 8, a CPU 11, a DRAM 12, amemory 13, a flash memory 14, an image display portion 15, a key inputportion 16, a card I/F 17, and a memory card 18.

The imaging lens 2 includes a focus lens, a zoom lens, etc. The lensdrive portion 3 is connected to the imaging lens 2. The lens driveportion 3 is configured by motors, a focus motor driver and a zoom motordriver. The motors drive the focus lens and the zoom lens forming theimaging lens 2, in the directions of their optical axes respectively.The focus motor driver and the zoom motor driver drive the focus motorand the zoom motor in accordance with control signals from the CPU 11,respectively.

The aperture 4 includes a not-shown drive circuit, which operates theaperture 4 in accordance with a control signal sent from the CPU 11. Theaperture 4 controls the quantity of light entering through the imaginglens 2. By the CCD (imaging device) 5, light of a subject projectedthrough the imaging lens 2 and the aperture 4 is converted into anelectric signal which is outputted as an imaging signal to the unitcircuit 7. In addition, the CCD 5 is driven in accordance with a timingsignal of a predetermined frequency generated by the TG 6.

The unit circuit (CDS/AGC/AD) 7 is configured by a Correlated DoubleSampling (CDS) circuit, an Automatic Gain Control (AGC) circuit and anA/D converter. The CDS circuit performs correlated double sampling onthe imaging signal outputted from the CCD 5, and holds the sampledimaging signal. The AGC circuit performs automatic gain control on thesampled imaging signal. The A/D converter converts an analog imagingsignal subjected to the automatic gain control into a digital signal.The imaging signal of the CCD 5 is sent as the digital signal to theimage processing portion 8 through the unit circuit 7. The unit circuit7 is driven in accordance with a timing signal of a predeterminedfrequency generated by the TG 6.

The image processing portion 8 performs image processing (pixelinterpolation processing, γ-correction, generation of luminance andcolor-difference signals, white balance processing, exposurecompensation processing, etc.) on image data sent from the unit circuit7, processing for compressing/expanding (for example,compressing/expanding in a JPEG format, an M-JPEG format or an MPEGformat) the image data, processing for synthesizing taken images, etc.The image processing portion 8 is driven in accordance with a timingsignal of a predetermined frequency generated by the TG 6.

The CPU 11 is a one-chip microcomputer for controlling the respectiveportions of the digital camera 1. Particularly in the first embodiment,the CPU 11 recognizes faces of subjects in two taken images shot withina predetermined time, estimates positions of the faces, and synthesizesa facial part (or the whole body of a subject) of one of the takenimages with the other taken image.

The DRAM 12 is used as a buffer memory for temporarily storing the imagedata taken by the CCD 5 and then sent to the CPU 11. The DRAM 12 is alsoused as a working memory for the CPU 11. In the memory 13, programsrequired for control of the respective portions of the digital camera 1by the CPU 11 and data required for the control of the respectiveportions are recorded. The CPU 11 performs processing along theprograms. The flash memory 14 or the memory card 18 serves as arecording medium for storing image data taken by the CCD 5, synthesizedimage data, etc.

The image display portion 15 includes a color LCD and a drive circuitthereof. When the digital camera 1 stands by for shooting, an image of asubject taken by the CCD 5 is displayed as a live view image on theimage display portion 15. For reproducing a recorded image, the recordedimage read from the flash memory 14 or the memory card 18 and expandedis displayed on the image display portion 15. The key input portion 16includes a plurality of operation keys such as a shutter SW, a zoom SW,a mode key, a SET key, cross keys, etc. The key input portion 16 outputsan operation signal to the CPU 11 in accordance with a user's keyoperation. The memory card 18 is removably attached to the card I/F 17through a not-shown card slot in a body of the digital camera 1.

The CPU 11 executes various processings in accordance with the programsstored in the memory 13.

Those processings may be achieved by the CPU 11 executing the programsstored in the memory 13. Alternatively, a dedicated processing circuit(processing portion) may be provided for performing the processings.

That is, the digital camera 1 includes:

an imaging portion which takes images of subjects;

an instruction portion which gives an instruction to synthesize twotaken images;

a first specification portion which specifies, as a first taken image, ataken image first shot by the imaging portion after the instructiongiven by the instruction portion or a taken image last shot by theimaging portion before the instruction given by the instruction portion;

a second specification portion which specifies a second taken imagehaving a predetermined relation to the first taken image; a facial imageextraction portion which extracts a facial image from one taken image ofthe two taken images including the first taken image and the secondtaken image; and

a synthesizing portion which synthesizes the facial image extracted bythe facial image extraction portion with a region of the other takenimage of the two taken images where there is no facial image.

Further, the second specification portion may specify, as the secondtaken image, a taken image shot in series by the own digital camera 1before or after the first taken image is taken.

Further, the second specification portion may specify, as the secondtaken image, a taken image shot by other digital camera before or afterthe first taken image is taken;

the digital camera 1 may further include a transmission portion whichtransmits the first taken image shot by the own digital camera 1, to theother digital camera; and

the two taken images including the first taken image and the secondtaken image may be synthesized on the other digital camera to which thefirst taken image is transmitted.

Further, the other digital camera may be located around the digitalcamera 1,

the digital camera 1 may further include a cooperative shootingrequesting portion which requests the other digital camera to performcooperative shooting, and

the second specification portion may specify, as the second taken image,a taken image shot by the other digital camera in response to therequest for the cooperative shooting made by the cooperative shootingrequesting portion.

Further, the facial image extraction portion may extract a facial imagefrom the first taken image taken by the imaging portion; and

the transmission portion may transmit the facial image extracted by thefacial image extraction portion to the other digital camera.

Further, the facial image extraction portion may extract a facial imagefrom the first taken image taken by the imaging portion; and thetransmission portion may transmit the facial image extracted by thefacial image extraction portion to the other digital camera.

Further, the instruction portion may give an instruction to synthesizethe two taken images, in response to the request for the cooperativeshooting received from the other digital camera which is located aroundthe digital camera 1; and

the first specification portion which has received the request for thecooperative shooting from the other digital camera which is locatedaround the digital camera 1 may specify the taken image shot by thedigital camera 1 as the first taken image and specify the taken imageshot by the other digital camera, which has requested the cooperativeshooting, as the second taken image.

Further, the facial image extraction portion may determine that aninstruction operation has been performed to give an instruction tosynthesize the two taken images, and extract a facial image from onetaken image of the two taken images including the first taken image shotby the digital camera 1 and the second taken image shot by the otherdigital camera which is located around the digital camera 1 when therequest for the cooperative shooting is received from the other digitalcamera; and

the synthesizing portion may synthesize the facial image extracted bythe facial image extraction portion with the region of the other takenimage of the two taken images where there is no facial image.

Further, the digital camera 1 may further include a determinationportion which determines whether predetermined synthesizing condition issatisfied or not before the synthesizing portion synthesizes images, and

the synthesizing portion may synthesize the facial image extracted bythe facial image extraction portion with the region of the other takenimage of the two taken images where there is no facial image when thedetermination portion determines that the predetermined synthesizingcondition is satisfied.

Further, the predetermined synthesizing condition may include that thereis one face which is not smaller than a predetermined size within theone taken image and there are a plurality of faces each of which is notsmaller than a predetermined size within the other taken image.

Further, the predetermined synthesizing conditions may include thatmembers whose faces are detected from one of the two taken images aredifferent from members whose faces are detected from the other takenimage.

Further, when the two taken images have a common background, thesynthesizing portion may synthesize the facial image of the one takenimage with the region of the other taken image where there is no facialimage so that a positional relation between the background and personsin each of the taken images can be kept.

Further, the digital camera 1 may further include a comparison portionwhich compares numbers of facial images contained in the two takenimages with each other,

the facial image extraction portion may extract facial images from thetaken image having a smaller number of faces based on a result ofcomparison in the comparison portion; and

the synthesizing portion may synthesize the facial images extracted bythe facial image extraction portions with a region of the taken imagehaving a larger number of faces, in which region there is no facialimage.

Further, the digital camera 1 may further include a storage portionwhich stores the two taken images specified by the specification portionso that the taken images are associated with each other.

FIGS. 2A to 2C are conceptual views showing a fundamental flow to take agroup photo with the digital camera according to the first embodiment.First, as shown in FIG. 2A, just after an instruction of a compositeshooting mode is given, an image a of other photographed persons istaken in accordance with a user operation (pushing down a shutterbutton) of a photographer. Next, as shown in FIG. 2B, the digital camera1 is handed to one of the other photographed persons, who performs auser operation to take an image b of the aforementioned photographer.When predetermined synthesizing conditions (which will be describedlater) are satisfied, a facial image included in one of the two takenimages a and b shot in series in accordance with the user operationsperformed in series within a predetermined period of time required forhanding over the camera is synthesized with the other taken image sothat a new composite image c is created, as shown in FIG. 2C. Thus, agroup photo in which all the persons have been photographed can besynthesized easily.

The aforementioned predetermined period of time is a period until theimage b is taken by one of the photographed persons with the digitalcamera 1 handed to the photographed person after the image a is taken.For example, the predetermined period of time may be assumed to be in arange of from about 3 seconds to about 30 seconds. Although two takenimages shot in series may be set as targets of synthesizing processingsimply, an upper limit or a lower limit of a shooting interval may beset as the aforementioned predetermined period of time in advance. Onthis occasion, when the shooting interval is shorter than the lowerlimit of the predetermined period of time or when the next image is nottaken within (the upper limit of) the predetermined period of time, thefirst image a is regarded as being taken normally, and the synthesizingprocessing is prevented from being performed. In other words, two takenimages shot in series within the predetermined period of time arespecified as two taken images which should be synthesized.

In addition, in the case where the synthesizing conditions are satisfiedto perform the synthesizing processing or the case where thesynthesizing processing is not performed, the taken image a and thetaken image b shot in series in accordance with the user operationsperformed in series within the predetermined period of time immediatelyafter the instruction of the aforementioned composite shooting mode isgiven are regarded as relevant to each other. Thus, the taken image aand the taken image b are associated with each other. Here, a methodsuch as addition of a tag to data may be used as the method forassociating the taken images. For example, when a photographer takes aphotographer's own image after taking a group photo, those two takenimages are associated with each other. Thus, the photographer canconfirm easily later that the photographer was there at that time.

FIG. 3 is a flow chart for explaining operation of the digital cameraaccording to the first embodiment. First, a shooting mode is inputted bya user operation (Step S10). Next, determination is made as to whetherthe inputted shooting mode is a composite shooting mode or not (StepS12). When the shooting mode is not a composite shooting mode,determination is made that the shooting mode is another shooting mode,e.g. a normal shooting mode, and normal shooting processing is executed(Step S14). After that, the processing is terminated (or returns to StepS10).

On the other hand, when the inputted shooting mode is a compositeshooting mode, an image a is taken with the digital camera 1 inaccordance with a user operation of a photographer (Step S16). Next,face recognition is performed on the taken image a, and positions offaces are estimated in the taken image a based on the result of the facerecognition (Step S18). Next, an image b is taken in accordance with auser operation of a photographer (Step S20). Face recognition isperformed on the taken image b, and a position of a face is estimated inthe taken image b based on the result of the face recognition (StepS22). That is, the taken images a and b are specified as two takenimages which were taken in series and which should be synthesized.

Next, determination is made as to whether synthesizing conditions aresatisfied or not (Step S24). The synthesizing conditions include whetheror not one face not smaller than a predetermined size is present in oneof the taken images (for example, the taken image b) and a plurality offaces each not smaller than a predetermined size are present in theother taken image (for example, the taken image a) as shown in FIG. 4(Step S24 a), or whether members whose faces are detected in the twotaken images a and b respectively are different or not as shown in FIG.5 (Step S24 b). When the synthesizing conditions are set thus,unnecessary synthesizing processing such as synthesizing taken imageswith no face image or synthesizing taken images with the same memberscan be prevented from being carried out.

When the synthesizing conditions are satisfied, a facial portion in onetaken image (for example, the taken image b) is detected while aposition where the facial portion will be embedded in the other takenimage (for example, the taken image a) is determined in order tosynthesize the facial portion (Step S26). Details of the processing inStep S26 in which a facial portion is detected in one taken image and aposition where the facial portion will be embedded in the other takenimage is determined will be described later.

Next, the facial portion of the one taken image (for example, the takenimage b) is synthesized in the position where the facial portion will beembedded in the other taken image (for example, the taken image a) sothat a composite image c is created and stored (Step S28). Next, thetaken image a and the taken image b are associated with each other (StepS30). After that, the processing is terminated (or returns to Step S10).

On the other hand, when the synthesizing conditions are not satisfied inStep S24, the taken image a and the taken image b are not subjected tosynthesizing processing but associated with each other in Step S30.After that, the processing is terminated (or returns to Step S10).

FIG. 6 is a flow chart for explaining the processing of Step S26according to the first embodiment, in which each facial portion isdetected in one taken image and a position where the facial portion willbe embedded in the other taken image is determined. FIGS. 7 and 8 areconceptual views showing an example of operation for deciding aninsertion position in a taken image a. The following description will bemade on the assumption that the one taken image is a taken image b andthe other taken image is a taken image a.

First, contents of predetermined settings are read (Step S40). Thecontents of the settings include designation of a range where the imageof the facial portion of the taken image b is not allowed to be embeddedin left, right, upper and lower edges of the taken image a(corresponding to values a and b shown in FIG. 7, for instance) becauseit looks poorly if the image of the facial portion of the taken image bis embedded exactly in an edge of the taken image a, setting of maximumnumbers of vertical and horizontal dots of the image to be embedded andthe maximum ratio of the image to be embedded to the taken image a, etc.

Next, a facial range to be inserted is determined based on the facialportion recognized in the taken image b (Step S42). The shaded portionshown in FIG. 7 corresponds to the facial range to be inserted. Next,the portion is expanded or reduced to an appropriate size (x, y)correspondingly to the size of each face in the taken image a based onthe contents of the settings, so as to be created as a to-be-insertedfacial image of the taken image b (Step S44). Next, the to-be-insertedfacial image of the taken image b is superimposed on the taken image a,and an insertion possible position for adding the to-be-inserted facialimage (of the photographer) obtained from the taken image b is extractedfrom the taken image a (Step S46).

Next, as shown in FIG. 7, first, the to-be-inserted facial image is puton the upper left of the taken image a. When there is no facial image ofthe taken image a (or no to-be-inserted facial image which has beenalready superposed) in the range where the to-be-inserted facial imagehas been put, the to-be-inserted facial image is displaced downward andmoved from right to left. Thus, a position just before a position wherea facial image of the taken image a (or a to-be-inserted facial imagewhich has been already superposed) is present at the beginning isdetermined as a position where the to-be-inserted facial image of thetaken image b will be inserted in the taken image a (Step S48).

For example, as shown in FIG. 8, assume that there are persons A, B andC in the taken image a. In this case, a position where theto-be-inserted facial image of the taken image b is not superposed onany person A, B or C of the taken image a is determined as the insertionposition. At this time, the position where the to-be-inserted facialimage of the taken image b will be inserted on the taken image a isdetermined, so that the remaining insertion possible positions in thetaken image a do not include the determined insertion position.

Next, determination is made as to whether all the facial images of thetaken image b have been inserted or not (Step S50). When there is afacial image which has not yet been inserted, the processing returns toStep S46, in which the aforementioned processing is repeated so that aposition where the to-be-inserted facial image of the taken image b willbe inserted is determined as one of the remaining insertion possiblepositions in the taken image a. It is a matter of course that theposition where each to-be-inserted facial image will be inserted isdetermined so that the to-be-inserted facial image can be prevented frombeing put on any facial image of the taken image a or any otherto-be-inserted facial image.

According to the aforementioned first embodiment, when there is carriedout an instruction operation for giving an instruction to synthesize twotaken images, two taken images shot in series in accordance with useroperations (pushing the shutter button) performed in series with onedigital camera at different timings, i.e. within a predetermined periodof time, are specified and the specified two taken images aresynthesized. In this manner, a group photo of a plurality of personsincluding a photographer can be taken easily.

In addition, when a plurality of faces each not smaller than apredetermined size are detected in one taken image and one face notsmaller than a predetermined size is detected in the other taken image,two taken images shot in accordance with user operations (pushing theshutter button) performed in series with one digital camera at differenttimings, i.e. within a predetermined period of time are specified, andthe specified two taken images are synthesized so that a group photo ofa plurality of persons including a photographer can be taken more surelyand easily.

In addition, when members whose faces are detected in two taken imagesare different, two taken images shot in accordance with user operations(pushing the shutter button) performed in series with one digital cameraat different timings, i.e. within a predetermined period of time, arespecified and the specified two taken images are synthesized so that agroup photo of a plurality of persons including a photographer can betaken efficiently and easily because two taken images with the samemembers are not synthesized with each other.

B. Second Embodiment

Next, a second embodiment of the invention will be described.

The second embodiment will be described on the assumption that a groupphoto is intended to be taken with the same scenery (such as a mountain,an ocean or a building) as the background in a journey etc. Therefore,the second embodiment has the following characteristic. That is, whentaken images a and b shot in accordance with user operations carried outin series within a predetermined period of time have a commonbackground, a position where a to-be-inserted facial image of one takenimage will be inserted in the other taken image is determined to keepthe positional relation between the background and each person in eachtaken image a, b. When the taken images a and b do not have a commonbackground, a position where a to-be-inserted facial image of one takenimage will be inserted in the other taken image is determined to preventfaces from being superposed on each other, in the same manner as in theaforementioned first embodiment. Since the configuration of the digitalcamera 1 is similar to that in FIG. 1, description thereof will beomitted.

FIG. 9 is a flow chart for explaining operation of the digital cameraaccording to the second embodiment. Steps similar to those in FIG. 3 arereferred to by the same numerals respectively. FIG. 9 is the same asFIG. 3 except Step S24 c and Steps S26 a and 26 b.

In the same manner as in the aforementioned first embodiment, faces intaken images a and b shot in accordance with user operations carried outin series within a predetermined period of time are recognized andpositions of the faces are estimated in Steps S10 to S22 immediatelyafter a composite shooting mode is inputted as a shooting mode. Afterthat, determination is made as to whether the taken images a and b havea common background or not (Step S24 c). That is, the taken images a andb are specified as two taken images shot in series. When the takenimages a and b have a common background, the position where ato-be-inserted facial image of one taken image will be inserted in theother taken image is determined to keep the positional relation betweenthe background and each person in each taken image a, b (Step S26 a).

On the contrary, when the taken images a and b do not have a commonbackground, a position where a to-be-inserted facial image of one takenimage will be inserted in the other taken image is determined to preventfaces from being superposed on each other, in the same manner as in theaforementioned first embodiment (Step S26 b).

After that, in the same manner as in the aforementioned firstembodiment, the facial portion of one of the taken images (for example,the taken image b) is synthesized with the other taken image (forexample, the taken image a) in the position where the facial portionwill be embedded therein, so that a composite image c is created andstored. The taken images a and b are associated with each other in StepS30, and the processing is terminated (or returns to Step S10).

Although Step S24 c and Steps 26 a and 26 b shown in FIG. 9 have beenexplained briefly, the method for determining the insertion position maybe fundamentally conform to the aforementioned flow chart shown in FIG.6.

According to the aforementioned second embodiment, when two taken imagesshot in accordance with user operations performed in series at differenttimings, i.e. within a predetermined period of time, have a commonbackground, the two taken images are synthesized while keeping thepositional relation between the background and each person. Thus, agroup photo of a plurality of persons including a photographer can betaken easily on the common background while keeping the positionalrelation between the background and each person.

C. Third Embodiment

Next, a third embodiment of the invention will be described.

The third embodiment has the following characteristic. That is, thenumber of faces contained in each taken image a, b is detected, andfacial images are cut out from the taken image with a smaller number offaces and synthesized with the other taken image having a larger numberof faces. Thus, processing for cutting out facial images andsynthesizing the facial images can be reduced. Since the configurationof the digital camera 1 is similar to that in FIG. 1, descriptionthereof will be omitted. In addition, since a main process is the sameas that in the first embodiment (FIG. 3) or the second embodiment (FIG.9), description thereof will be omitted.

FIG. 10 is a flow chart for explaining processing according to the thirdembodiment, in which each facial portion is detected in one taken imageand a position where the facial portion will be embedded in the othertaken image is determined. First, contents of predetermined settings areread (Step S60). The contents of the settings include designation of arange where any image to be embedded is not allowed to be embedded inthe left, right, upper and lower edges, setting of maximum numbers ofvertical and horizontal dots of the image to be embedded or the maximumratio of the image to be embedded to a taken image with which the imageto be embedded will be synthesized, etc.

Next, the number of recognized faces contained in a taken image a isdetected (Step S62), and the number of recognized faces contained in ataken image b is detected (Step S64). Next, a facial range to beinserted is determined from a taken image whose number of faces issmaller (Step S66), and a to-be-inserted facial image of the taken imagewhose number of faces is smaller is created (Step S68). On thisoccasion, the to-be-inserted facial image may be expanded or reduced toan appropriate size correspondingly to the size of each face in theother taken image, whose number of faces is larger, based on thecontents of the settings. Next, the to-be-inserted facial image issuperimposed on the taken image whose number of faces is larger, and aposition where the to-be-inserted facial image can be inserted isextracted from the taken image whose number of faces is larger, in orderto synthesize the to-be-inserted facial image (Step S70).

Next, the to-be-inserted facial image is put on the upper left of thetaken image whose number of faces is larger. When there is no facialimage of the taken image whose number of faces is larger (or noto-be-inserted facial image which has been already superposed) in therange where the to-be-inserted facial image has been put, theto-be-inserted facial image is displaced downward or moved from right toleft. Thus, a position just before a position where a facial image ofthe taken image whose number of faces is larger (or a to-be-insertedfacial image which has been already superposed) is present at thebeginning (or a position where the to-be-inserted facial image is notsuperposed on any facial image) is determined as a position where theto-be-inserted facial image of the taken image whose number of faces issmaller will be inserted among the facial images of the taken imagewhose number of faces is larger (Step S72).

Next, determination is made as to whether the facial images of the takenimage whose number of faces is smaller have been inserted or not (StepS74). When there is any facial image which has not yet been inserted,the flow of processing returns to Step S70, in which the aforementionedprocessing is repeated to determine an insertion possible position wherethe to-be-inserted facial image of the taken image whose number of facesis smaller will be inserted in the taken image whose number of faces islarger. It is a matter of course that the insertion positions aredetermined so that each to-be-inserted facial image can be preventedfrom being superposed on any facial image of the taken image a or anyother to-be-inserted facial image which has been already inserted.

According to the aforementioned third embodiment, the number of facescontained in each of two taken images is detected, a facial portion iscut out from the taken image with a smaller number of faces andsynthesized with the taken image with a larger number of faces.Accordingly, a group photo of persons including a photographer can betaken efficiently and easily.

According to the aforementioned first to third embodiments, a takenimage a and a taken image b shot in series in accordance with useroperations performed in series within a predetermined period of timeimmediately after a composite shooting mode is inputted as a shootingmode are specified as two taken images shot in series to be synthesized,and the specified two taken images a and b are synthesized. Theinvention is not limited thereto. Alternatively, taken images a and bmay be specified as two taken images shot in series to be synthesizedand the specified two taken images a and b may be synthesized when thecomposite shooting mode is inputted immediately after the taken image aand the taken image b are shot in series in accordance with useroperations performed in series within a predetermined period of time. Inthis case, when the composite shooting mode is not inputted, the takenimages a and b are recorded and stored as individual taken imageswithout being synthesized, in the same manner as in normal shooting.

Shooting immediately after the composite shooting mode is inputted (orinputting the composite shooting mode immediately before shooting) meansthat shooting is carried out in accordance with user operationsperformed in series within a predetermined period of time before thedigital camera is powered off or the operation mode is switched from thecomposite shooting mode to another operation mode after the compositeshooting mode is inputted. On the other hand, inputting the compositeshooting mode immediately after the taken image a and the taken image bare taken in series (or taking the taken image a and the taken image bimmediately before the composite shooting mode is inputted) means thatthe composite shooting mode is inputted before another image is taken,the digital camera is powered off or the operation mode is switched fromthe composite shooting mode to another operation mode after the takenimage a and the taken image b are taken in accordance with useroperations carried out in series within a predetermined period of time,or the composite shooting mode is inputted before the elapse of apredetermined period time after the taken image a and the taken image bare taken in accordance with user operations carried out in serieswithin a predetermined period of time.

D. Fourth Embodiment

Next, a fourth embodiment of the invention will be described.

FIG. 11 is a block diagram showing configuration of a digital cameraaccording to the fourth embodiment. Parts corresponding to those in FIG.1 are referred to by the same numerals respectively, and descriptionthereof will be omitted. In FIG. 11, a communication portion 19communicates with another digital camera located in the surrounding,based on short-distance wireless communication (infrared communication,Bluetooth, and so on). Particularly in the fourth embodiment, thecommunication portion 19 is used for taking an image in cooperation withanother digital camera located in the surroundings.

FIGS. 12A to 12E are conceptual views showing a fundamental flow to takea group photo with digital cameras according to the fourth embodiment.First, when an instruction of a cooperative shooting mode is given, thecooperative shooting mode is set between a digital camera 1-1 and adigital camera 1-2 as shown in FIG. 12A. Next, in accordance with a useroperation of a photographer, an image a of other photographed persons istaken with the digital camera 1-1 as shown in FIG. 12B, and the digitalcamera 1-1 notifies the other digital camera 1-2 of completion of theshooting as shown in FIG. 12C. Next, in accordance with a user operationof one of the other photographed persons, an image b of the previousphotographer is taken with the digital camera 1-2 which has received thenotification indicating completion of the shooting as shown in FIG. 12D,and the digital camera 1-2 notifies the digital camera 1-1 of completionof the shooting as shown in FIG. 12E. That is, the taken images a and bare specified as two taken images shot in series.

After that, the taken image a, b shot with one of the digital cameras1-1 and 1-2 is transmitted to and received from the other digitalcamera. When predetermined synthesizing conditions are satisfied, afacial image contained in one of the taken images a and b shot in seriesin accordance with the user operations performed in series within apredetermined period of time is synthesized with the other taken imageso that a new composite image is created. Thus, it is possible to easilysynthesize a group photo in which all the persons have beenphotographed.

In addition, either when the synthesizing conditions are satisfied andsynthesizing processing is performed or when the synthesizing processingis not performed, the taken image a and the taken image b shot in seriesin accordance with the user operations with the digital cameras 1-1 and1-2 respectively immediately after the instruction of the cooperativeshooting mode is given are regarded as relevant so that the taken imagea and the taken image b are associated with each other. Here, a methodsuch as addition of a tag to data may be used as the method forassociating the taken images. For example, when a photographer takes animage of the photographer after taking a group photo, the two takenimages are associated with each other. Thus, the photographer can easilyconfirm later that the photographer was there at that time.

FIG. 13 is a flow chart for explaining operations of the digital camerasaccording to the fourth embodiment. FIG. 14 is a sequence diagram forexplaining operations of digital cameras 1-1 and 1-2 according to thefourth embodiment. First, a shooting mode is inputted by a useroperation on one of the digital cameras (Step S80). Next, determinationis made as to whether the inputted shooting mode is a cooperativeshooting mode or not (Step S82). When the inputted shooting mode is nota cooperative shooting mode, determination is made that the shootingmode is another shooting mode, for example, a normal shooting mode, andnormal shooting processing is executed (Step S88). After that, theprocessing is terminated (or returns to Step S80).

On the other hand, when the inputted shooting mode is a cooperativeshooting mode, the communication portion 19 uses a predeterminedcommunication method to request a digital camera, which has the samefunction and may be located in the surroundings, to perform cooperativeshooting (Step S84, see FIG. 12A). The other digital camera 1-2 islocated around the digital camera 1-1, and the other digital camera 1-2responds to the request of the cooperative shooting. More specifically,on receiving the request for cooperative shooting, for example, thedigital camera 1-2 informs a user by displaying a message indicating therequest. The user confirms that the request for cooperative shooting hasbeen received, and gives an instruction through the key input portion 16or the like as to whether to perform cooperative shooting or not.

Whether the other digital camera 1-2 responds to the request forcooperative shooting from the digital camera 1-1 or not may bedetermined based on whether setting to respond to the request forcooperative shooting has been made on the digital camera 1-2 in advanceor not. Alternatively, ID information of one or more digital cameraswhich may be paired with the digital camera 1-2 may be registered inadvance so that the digital camera 1-2 can respond to only the requestfor cooperative shooting from each digital camera having a registeredID.

The digital camera 1-1 determines whether there is a response from adigital camera in the surroundings or not, that is, whether a digitalcamera which can perform cooperative shooting is in the surroundings ornot (Step S86). When no digital camera which can perform cooperativeshooting is in the surroundings, the digital camera 1-1 determinesnormal shooting, and executes normal shooting processing (Step S88).After that, the processing is terminated (or returns to Step S80).

On the other hand, when a digital camera which can perform cooperativeshooting is in the surroundings (for example, there is a response fromthe digital camera 1-2), the digital camera 1-1 transmits settingconditions for cooperative shooting to the partner digital camera 1-2(Step S90). For example, it may be conceived that the setting conditionsfor the cooperative shooting include which digital camera to performsynthesizing processing, from which digital camera to which digitalcamera to transmit an image, whether to transmit only a cut-out facialimage to the other digital camera in order to reduce the capacity fortransmission of the image, whether to transmit the whole of a takenimage to the other digital camera, whether to transmit a taken image toa server on a network to perform image synthesizing on the server, etc.The following description will be made in the case where the digitalcameras 1-1 and 1-2 exchange taken images for each other and performimage synthesizing.

Next, determination is made as to whether an OK signal indicating thatthe setting conditions have been confirmed and set has been receivedfrom the partner digital camera 1-2 or not (Step S92). When the OKsignal has been received, the digital camera 1-1 notifies a user ofreadiness for shooting on the image display portion 15 or by sound. Inaccordance with an operation performed on the shutter button by the userwho has confirmed the notification, the digital camera 1-1 takes animage a (Step S94, see FIG. 12B). After that, the digital camera 1-1transmits a message indicating completion of the shooting to the partnerdigital camera 1-2 (Step S96, see FIG. 12C).

On receiving the message indicating completion of the shooting, thepartner digital camera 1-2 notifies a user of readiness for shooting onthe image display portion 15 or by sound. In accordance with anoperation performed on the shutter button by the user who has confirmedthe notification, the digital camera 1-2 takes an image b (see FIG. 12Dand Step S200 in FIG. 14). After that, the digital camera 1-2 transmitsa message indicating completion of the shooting to the digital camera1-1 (Step S96, see FIG. 12E).

The digital camera 1-1 determines whether the message indicatingcompletion of the shooting has been received from the partner digitalcamera 1-2 or not (Step S98). On receiving the message indicatingcompletion of the shooting, the digital camera 1-1 controls thecommunication portion 19 to transmit the taken image a to the partnerdigital camera 1-2 (Step S100). The digital camera 1-2 receives thetaken image a from the digital camera 1-1. In the same manner, onreceiving the taken image a from the digital camera 1-1, the digitalcamera 1-2 transmits the taken image b to the digital camera 1-1 (seeStep S202 in FIG. 14). The digital camera 1-1 receives the taken image bfrom the partner digital camera 1-2 (Step S102). That is, the takenimages a and b are specified as two taken images shot in series.

The digital camera 1-1 determines whether synthesizing conditions aresatisfied or not in the same manner as in the aforementioned firstembodiment (Step S104). When the synthesizing conditions are satisfied,a facial portion of one taken image (for example, the taken image b) isdetected, and a position where the facial portion will be embedded inthe other taken image (for example, the taken image a) is determined inorder to synthesize the facial portion (Step S106).

Next, the facial portion of the one taken image (for example, the takenimage b) is synthesized with the position where the facial portion willbe embedded in the other taken image (for example, the taken image a) sothat a composite image c is created and stored (Step S108). Next, thetaken image a and the taken image b are associated with each other (StepS110). After that, the processing is terminated (or returns to StepS80).

On the other hand, when the synthesizing conditions are not satisfied inStep S106, the taken image a and the taken image b are not subjected tosynthesizing processing but associated with each other in Step S110.After that, the processing is terminated (or returns to Step S80).

When setting is made in the setting conditions of the cooperativeshooting so as to allow the digital camera 1-2 to perform synthesizingprocessing, the digital camera 1-2 also synthesizes a facial portion ofone taken image (for example, the taken image b) with a position wherethe facial image will be embedded in the other taken image (for example,the taken image a) as described above so that a composite image c iscreated and stored. The taken image a and the taken image b areassociated with each other, and the processing is terminated (see StepS203 in FIG. 14).

Although the aforementioned fourth embodiment has been described in thecase where the digital cameras 1-1 and 1-2 can synthesize imagesindividually, only one of the digital cameras may perform synthesizingprocessing according to some setting conditions of cooperative shootingor the taken images a and b may be transmitted to a server on a networkat a timing desired by a user after the images are exchanged, so thatsynthesizing processing can be performed on the server.

In addition, in the fourth embodiment, the aforementioned predeterminedperiod of time is a waiting time between the time when the digitalcamera 1-1 takes an image a while requesting the digital camera 1-2 toperform cooperative shooting and the time when the digital camera 1-2takes an image b in response to the request for the cooperativeshooting. For example, it is assumed that a preparatory time intervalbefore a photographer who performs first shooting puts down a camera andmakes a pose after the shooting is in a range of from about one to aboutten seconds.

In addition, although the fourth embodiment has been described in thecase where the digital camera 1-2 takes the image b in accordance with auser operation, the invention is not limited thereto. Automatic shootingmay be performed when a predetermined period of time has passed.

According to the aforementioned fourth embodiment, only when a pluralityof persons take images of each other with their own digital camerasrespectively, a group photo of the persons including photographers canbe easily taken as if the persons were photographed simultaneously witha single camera.

In addition, when synthesizing conditions includes whether to allow onlyone digital camera to transmit a taken image or whether to transmit onlya cut-out facial image to a digital camera which will synthesize thefacial image, it is possible to reduce the capacity for transmission ofthe image. Thus, a group photo of a plurality of persons includingphotographers can be taken more efficiently and more easily.

What is claimed is:
 1. An imaging apparatus comprising: an imagingportion that takes images of subjects; a first determination portionthat, when two taken images taken by the imaging portion are to besynthesized, compares (i) a number of facial images contained in onetaken image of the two taken images with (ii) a number of facial imagescontained in the other taken image of the two taken images, anddetermines which taken image has a smaller number of facial images; anda synthesizing portion that, after the determination by the firstdetermination portion, extracts a facial image from the taken image ofthe two taken images determined to have the smaller number of facialimages, and synthesizes the extracted facial image with a region of theother taken image of the two taken images where there is no facialimage.
 2. The imaging apparatus according to claim 1, furthercomprising: a second determination portion that determines, as asynthesizing condition, whether or not a facial image which is notsmaller than a predetermined size is contained in one taken image, andwhether or not a plurality of facial images each of which is not smallerthan a predetermined size are contained in another taken image, amongtwo taken images taken by the imaging portion; and a control portionthat controls the synthesizing portion to synthesize the two takenimages when the second determination portion determines that thesynthesizing condition is satisfied.
 3. An imaging apparatus comprising:an imaging portion that takes images of subjects; a synthesizing portionthat extracts a facial image from one taken image and that synthesizesthe extracted facial image with a region of another taken image wherethere is no facial image, among two taken images taken by the imagingportion; a determination portion that determines whether or not memberswhose facial images are contained in one taken image of the two takenimages are different from members whose facial images are contained inthe other taken image of the two taken images; and a control portionthat controls the synthesizing portion to synthesize the two takenimages when the determination portion determines that the members whosefacial images are contained in the one taken image are different fromthe members whose facial images are contained in the other taken image.4. An image synthesizing method in an imaging apparatus comprising animaging portion for taking images of subjects, the method comprising:when two taken images taken by the imaging portion are to besynthesized, comparing a number of facial images contained in one takenimage of the two taken images with a number of facial images containedin the other taken image of the two taken images; determining whichtaken image of the two taken images has a smaller number of facialimages; extracting a facial image from one taken image of the two takenimages determined to have the smaller number of facial images; andsynthesizing the extracted facial image with a region of the other takenimage of the two images where there is no facial image.
 5. The imagesynthesizing method according to claim 4, further comprising:determining, as a synthesizing condition, whether or not a facial imagethat is not smaller than a predetermined size is contained in one takenimage and whether or not a plurality of facial images each of which isnot smaller than a predetermined size are contained in another takenimage, among two taken images taken by the imaging portion; andsynthesizing the two taken images when it is determined that thesynthesizing condition is satisfied.
 6. An image synthesizing method inan imaging apparatus comprising an imaging portion for taking images ofsubjects, the method comprising: extracting a facial image from onetaken image among two taken images shot by the imaging portion;synthesizing the extracted facial image with a region of the other takenimage of the two taken images where there is no facial image; anddetermining whether or not members whose facial images are contained inone taken image of the two taken images are different from members whosefacial images are contained in the other taken image of the two takenimages; wherein the synthesizing is performed when it is determined thatthe members whose facial images are contained in the one taken image aredifferent from the members whose facial images are contained in theother taken image.